On the Theory of Learning with Privileged Information (Full version)

In Learning Using Privileged Information (LUPI) paradigm, along with the stan-dard training data in the decision space, a teacher supplies a learner with the priv-ileged information in the correcting space. The goal of the learner is to find aclassifier with a low generalization error in the decision space. We consider anempirical risk minimization algorithm, called Privileged ERM, that takes into ac-count the privileged information in order to find a good function in the decisionspace. We outline the conditions on the correcting space that, if satisfied, allowPrivileged ERM to have much faster learning rate in the decision space than theone of the regular empirical risk minimization.